Böck: Multiple vulnerabilities in RPM – and a rant
The development process of RPM seems to be totally chaotic, it's neither clear where one reports bugs nor where one gets the latest code and security bugs don't get fixed within a reasonable time. There's been some recent events that make me feel especially worried about this..." It seems that some of the maintenance issues with RPM may not have improved greatly since they were reported here ten years ago.
Posted Aug 29, 2016 15:13 UTC (Mon)
by rsidd (subscriber, #2582)
[Link] (124 responses)
I'd love to hear Red Hatter responses to this. My first exposure to linux was Red Hat, in the mid-late 1990s (there were slackware machines around but falling out of favour in my grad-school group). Later I discovered the Debian world, via Knoppix, and it was like night and day. I have not ventured back into the RPM world but people on LWN and elsewhere keep saying rpm and deb are basically equivalent, yum and apt are basically equivalent, and so on. On the other hand, people continue to mention "rpm-hell" and nobody has ever talked about "deb-hell" so I haven't been tempted back.
Böck: Multiple vulnerabilities in RPM – and a rant
But I can accept that a technology that seems inferior to me works better for some people -- maybe they experience dependency-hell on Debian and not in the RH family, or maybe there are other positives that outweigh this. But how does one accept something like this?
“Thanks for the report. We also received about 30+ crash reports in RPM from a different reporter recently so processing all of them (yours and the others) will take quite a bit of time. We simply don't have the resources to spend hours upon hours analyzing all crash reports.”How does an organization with Red Hat's resources not have time to analyse bug reports on the most fundamental building block of their distribution?
Posted Aug 29, 2016 15:39 UTC (Mon)
by NAR (subscriber, #1313)
[Link] (4 responses)
Posted Aug 29, 2016 16:01 UTC (Mon)
by bbockelm (subscriber, #71069)
[Link]
Some amount of parsing happens _before_ the signature check and it is unclear to the author whether any of the bugs could trigger during that phase. If so, it would be fairly unexpected to have arbitrary root-level code execution done as part of validating the RPM's signature.
*Note* though the author is careful to say this isn't necessarily the case and it is his opinion that the parser should be robust regardless.
Posted Aug 29, 2016 16:22 UTC (Mon)
by josh (subscriber, #17465)
[Link] (2 responses)
Posted Aug 31, 2016 18:35 UTC (Wed)
by ferringb (subscriber, #20752)
[Link] (1 responses)
If you can massage that process, then you can target the defenses the builder has in place to prevent archive attacks- things like symlink traversal to allow arbitrary writing to the builder's host OS (pre-VM). At that point, it's just a matter of choosing proper targets for overwriting- if they're using xen (and the unpack was done as root w/out any fakeroot depriving), you'd just go after the xen binary for example.
It's a bit twisty, but that what I'm describing above is an actual vulnerability that occurred for opensuse build service back in '10/'11; it doesn't have to be a remote code exploit against RPM for it to be problematic- it can just be breaking assumptions that the build infrastructure relies on, then using more traditional attacks.
I'd assume these pathways have been improved since I last took a whack at them; however, people continually make mistakes when it comes to unpacking things safely, so potential issues in things like rpm tooling shouldn't be treated lightly.
Posted Sep 3, 2016 7:05 UTC (Sat)
by jospoortvliet (guest, #33164)
[Link]
I don't know how modern distros develop other than how opensuse does it but I guess they all have some way where packages are built on an official server in a transparent way - a system where tens or hundreds of people could upload official packages is extremely easy to hack - break ONE system and you can upload exploits... it would be insane to trust such a distros for anything security related.
Posted Aug 29, 2016 15:46 UTC (Mon)
by smoogen (subscriber, #97)
[Link] (48 responses)
So please don't take this wrong, but you don't want to hear Red Hat's response. You want something to confirm that you have already made the right choice and that choosing RPM would be a mistake. So here is me as a paid employee of Red Hat saying this: You made the right choice for you. :)
Posted Aug 29, 2016 15:58 UTC (Mon)
by pboddie (guest, #50784)
[Link] (9 responses)
Since Python-centred package users tend to use "virtualenvs" all over the place these days, where each program has its own sandbox with its own precisely-defined set of packages, reports of package conflicts are probably less likely than from the Ruby crowd unless they've also gone down the same road. (Yes, virtualisation: the kind of solution which adds another layer to a stack that already offers such features - thanks to years of building up such functionality out of genuine need in a more economical way - and which solves every problem but ultimately no problem all at the same time.)
Posted Aug 29, 2016 16:15 UTC (Mon)
by ms-tg (subscriber, #89231)
[Link] (7 responses)
For those who are not aware, many (perhaps most) rubyists use the RVM "Gemsets" feature for this: https://rvm.io/gemsets/basics
Posted Aug 29, 2016 16:18 UTC (Mon)
by ms-tg (subscriber, #89231)
[Link] (6 responses)
Posted Aug 29, 2016 21:25 UTC (Mon)
by bronson (subscriber, #4806)
[Link] (5 responses)
Once you've done that, you're guaranteed that only those exact gem versions are ever used. It doesn't require RVM and it's easy to add to your project's version control. Update your dependencies at any time by running 'bundle update'.
It's pretty great. Check out the "Gemfile" and "Gemfile.lock" files on almost any Ruby project these days. It works so well that barely anybody uses gemsets anymore.
Posted Aug 30, 2016 14:31 UTC (Tue)
by jhoblitt (subscriber, #77733)
[Link]
Posted Aug 30, 2016 22:13 UTC (Tue)
by lsl (subscriber, #86508)
[Link] (3 responses)
Splendid. You've also made your users' live a living hell by making them hunt down *and* maintain those exact gem versions instead of just installing the ones from their distro. You've probably never tested anything else, so the attempt to fix up this misery is sure to result in yet more pain.
As a bonus, all these locked-down dependencies serve as the perfect excuse for the wider ruby ecosystem to just forego any kind of sane API design and hygiene. If you don't like the constant API breakage, you can just stay locked onto your random old version for all eternity, right?
I can see how things like Bundler can be very convenient for some developers. I just don't think it can compensate for the enormous downsides, no matter how convenient it may be.
Posted Sep 2, 2016 14:09 UTC (Fri)
by yoe (guest, #25743)
[Link] (2 responses)
Posted Sep 2, 2016 18:02 UTC (Fri)
by fuhchee (guest, #40059)
[Link]
Posted Sep 3, 2016 7:09 UTC (Sat)
by jospoortvliet (guest, #33164)
[Link]
Posted Aug 30, 2016 13:06 UTC (Tue)
by smoogen (subscriber, #97)
[Link]
Posted Aug 29, 2016 18:52 UTC (Mon)
by h2 (guest, #27965)
[Link] (37 responses)
As a developer, and distro programmer (support applications) a few times I expanded some of my projects to support rpm and pacman arch packaging, and to do that, I ended up doing a lot of installs for testing and debugging. pacman was a joke in terms of robustness, it's features are simplicity and speed, not robustness, but what I found in particularly irksome with rpm based distributions was the totally poor to non existent ability to upgrade from one release to the next, which has always been a core requirement for debian apt, in fact, apt is primarily designed to do this, which in turn governs the debian packaging rules, which are BY FAR the strictest in the linux distro package manager world. apt and deb packages work together to achieve this result, with fedora/redhat/suse this is just an afterthough, one that you hope you can finally get to work by releasing new tools like yum or whatever the latest one is called.
I realize that when you work for a corporation you have to drink the coolaid to get ahead or even manage to stay there, that's well understood and well known, but it's still sad to see it repeated time after time. More amusingly, I had a friend who is a senior admin at a major web firm, tons of properties, they were running primarily rpm, and moving towards deb systems, and when he started, I warned him about rpm hell, and it took him very little time to experience it himself. Since he had no bias in the matter, he's a unix guy by preference, he was just seeing what was there. He quickly found the superiority of apt, which is obvious, and likewise quickly grew to dislike rpm, for obvious reasons. Reasons redhat employees do not allow themselves to think.
Now, I'll have to go back to my never reinstalled debian boxes, in fact, just did a cross grade of an old 32 bit system to 64 bit, check in with me when you can do that with rpm, lol, and of course, all the systems are upgraded continuously, not perfect, of course, nothing is, particularly since debian is primarily run by free software developers who do it for free, and redhat, well, as you can see with this bug report, it's run by corporate types who do what they are told. That's the difference, that's why dpkg fixed the issue and in 8 days and why redhat guys made excuses, the expenses probably needed to be approved and sent up the ladder, the debian guys cared about their software and wanted the bug fixed. People who send me bug reports on my software, which I do as free software for free, will see them fixed in often hours, that's because I am free to do my stuff as I please, and I care.
The main reason, by the way, I don't contribute money to lwn.net, I really would in general, is because I'm so sick of its always pro whatever redhat is doing bias, it's boring and tedious, I can see corporate blindness anywhere I want, it's standard to drink the coolaid, if I talk to apple employees, they will be blind to how bad and buggy their software is, and how invasive their practices are, microsoft types will maintain that their operating system is a really good one, and redhat guys, well, they will insist rpm isn't a badly executed package manager. debian guys, well, you know, it's not a corporate enterprise, so in general, they just try to do the best job they have time and energy to do, sometimes it's enough, sometimes it's not, but I'll take that over corporate garbage any day of the week.
Posted Aug 29, 2016 19:43 UTC (Mon)
by pizza (subscriber, #46)
[Link]
FYI, "rpm hell" referred to the fact that RPM doesn't include a dependency resolver, so folks doing manual package installations/upgrades had to work this out for themselves.
dkpg doesn't have a dependcy resolver either -- so it was subject to the exact sort of "deb hell" as rpm. However, most folks just used apt instead of manually invoking dpkg by hand, which took care of all of the dependencies.
Similarly, once Red Hat formally adopted yum, "rpm hell" disappeared nearly immediately.
> I can see corporate blindness anywhere I want [...]
Don't forget that you're also subject to your own cognative blindness.
Posted Aug 29, 2016 19:45 UTC (Mon)
by niner (subscriber, #26151)
[Link] (2 responses)
Our servers started out with SuSE Linux 8.2 in 2003 and now run openSUSE 13.2. 13 years worth of distribution upgrades.
And yes, both my personal machines and the servers started out with i586 installations and were cross graded to x86_64 in ~ 2007. Back then Debian still had no multi-arch story at all. Because the "as good as possible" deb format didn't have the support.
Can't tell you how sick I am (to stick with to your words) of Debian fanboys who have nothing but unsubstantiated claims.
Posted Aug 29, 2016 20:33 UTC (Mon)
by pizza (subscriber, #46)
[Link] (1 responses)
I only did that total reinstall as I was consolidating two servers into one, and neither system's previous image booted successfully on the new hardware due to the significant hardware changes.
My personal systems have seen a lot more churn for various reasons (trying out different distros, changing employers, hardware experiments, portable vs non-portable boxes, etc etc).
Posted Sep 3, 2016 7:15 UTC (Sat)
by jospoortvliet (guest, #33164)
[Link]
Moreover, where debian is still stuck in 2000 with freezes and testing repositories, openSUSE has a real rolling release with fully automated testing and a modern collaborative packaging tool which follows a git style work flow. Debian is simply fossilised and it is clear why: folks like parent have not looked outside the debian ecosystem for 15 years and refuse to acknowledge that the rest of the world have moved on.
Posted Aug 29, 2016 20:24 UTC (Mon)
by corbet (editor, #1)
[Link] (16 responses)
Posted Aug 30, 2016 14:09 UTC (Tue)
by nix (subscriber, #2304)
[Link]
(Except, not, since this is all obviously totally ridiculous, but it's the only thing I can imagine that might satisfy the original borderline-troll's unsubstantiated accusations of bias. Sometimes one lot is mentioned more than the other lot because the first lot is *doing more work*, sigh.)
Posted Aug 30, 2016 15:45 UTC (Tue)
by cate (subscriber, #1359)
[Link]
Fedora and RedHat articles are more frequents (about design choices, flames about what to include on next releases, etc.). Debian and Ubuntu articles are less frequent, and usually the articles also compares what RedHat did (and it seems as "reference behaviour".
But more frequent articles about RedHat doesn't mean more positive article about RedHat.
Posted Aug 30, 2016 20:51 UTC (Tue)
by xtifr (guest, #143)
[Link] (13 responses)
(Wouldn't bother me much anyway, as I use RH professionally a lot—even though I have a mild preference for Debian for personal use—so I'm still interested in the Fedora stories. And for the record, I'm pretty good at putting together both rpms and debs, and I don't think either package format is noticeably better than the other.)
Posted Aug 30, 2016 21:14 UTC (Tue)
by corbet (editor, #1)
[Link] (12 responses)
Posted Aug 30, 2016 22:41 UTC (Tue)
by johannbg (guest, #65743)
[Link] (3 responses)
How are Arch,Gentoo,Mageia,OpenSuse more closed then Fedora or Debian?
What about upstream community themselves which make up those distributions, is the perception of those feeling closed as well?
Posted Aug 31, 2016 5:55 UTC (Wed)
by corbet (editor, #1)
[Link] (2 responses)
The openSUSE community certainly makes itself heard on such things, but the governance is different, more centralized.
Posted Aug 31, 2016 8:19 UTC (Wed)
by karkhaz (subscriber, #99844)
[Link]
> we usually don't announce things that are still being discussed or worked on. People who are interested in development and future plans should read arch-dev-public
that would never happen in Debian, where there was an enormous entire-community relatively democratic discussion about that, and it took forever to resolve. Arch, instead, is run by a cabal of a half-dozen developers and barely listen to the community; and most of us who use Arch like it that way :) the `official' explanation for moving to systemd is still this forum post [2].
[1] https://bbs.archlinux.org/viewtopic.php?pid=1148067#p1148067
Posted Sep 1, 2016 1:33 UTC (Thu)
by johannbg (guest, #65743)
[Link]
In the end of the day I think there exist only two primary distributions ( as in not derivatives ) that are truly community driven ( as in are not bound by the BD motel in one form or another thus arguably are the only one that can truly call themselves community driven distributions ) and those are Debian and Mageia which makes them the distribution in which individual contribution gain the most value of their contributed time.
If you and the rest of the writers here on lwn would have a saying how would you and the rest of the writers here envision project and distribution approaching media?
Posted Aug 31, 2016 6:04 UTC (Wed)
by mrdocs (guest, #21409)
[Link] (7 responses)
Having been a SUSE Enterprise customer first, then openSUSE member for more than 10 years and working for SUSE now for 4, I have to say how can we help our distinguished editor's knowledge of our community ?
I think partly SUSE (note SuSE has been deprecated some ten years ago), does not get the same attention being originally a German and very engineering focused company. It is changing, but as someone from the states, you do notice this tendency still manifests itself still. openSUSE marches to its own drumbeat and will on occasion make decisions which do not align strictly to SUSE business, but this not just tolerated, but encouraged.
In sum, an offer to connect you to the SUSE/openSUSE world :)
Posted Aug 31, 2016 12:27 UTC (Wed)
by smoogen (subscriber, #97)
[Link] (6 responses)
Please do not take this as a statement that Fedora is some sort of paragon that other OS's should emulate. We have plenty of warts and should do better.
Posted Aug 31, 2016 14:08 UTC (Wed)
by madscientist (subscriber, #16861)
[Link] (5 responses)
If you make a decision on private or semi-private lists and then publish that decision, then LWN will have one blurb article about the decision with a link to the announcement of the decision.
If you debate and discusses it on public lists with lots of input before making a decision, then LWN will have one blurb article about the decision with a link to the announcement, but it ALSO may have one or more full-length articles discussing the process of making the decision and providing summaries of the various positions and reasoning. Because in our F/OSS world the process of developing software and arriving at consensus is often at least as, if not more, interesting than the actual result of that process.
Posted Aug 31, 2016 16:21 UTC (Wed)
by smoogen (subscriber, #97)
[Link]
Posted Aug 31, 2016 18:36 UTC (Wed)
by rgmoore (✭ supporter ✭, #75)
[Link]
It's also worth considering that when those discussions turn into arguments- which is pretty often for the most contentious issues- the news about them can wind up sounding negative. Not all publicity is necessarily good publicity, and airing your decision making process with all its flaws is a good example.
Posted Sep 2, 2016 0:27 UTC (Fri)
by jschrod (subscriber, #1646)
[Link] (2 responses)
Decisions are also made and communicated very openly on these discussion lists.
Thus, the almost non-existant coverage on openSUSE developments has nothing to do with private or semi-private lists. It's all very public. In fact, that few to no reports about OBS exist -- the best thing that the openSUSE/SUSE community ever produced -- are sad.
And that's just about the community that I know about because I used it not long ago. I assume Gentoo or Arch folks can report similar stories.
Nevertheless, here's the best place to go for Linux News once a week, and there will be no better place. :-) :-)
Cheers, Joachim
[*] Please don't get me wrong. h2 is nuts. Or, more probably, since he doesn't even have a proper account, he's a plain troll. Here's to the LWN.net community that they respond to a troll with proper discussion about percieved or actual biases and about inviting responses if they really exist or if they are felt to exist.
Posted Sep 2, 2016 7:40 UTC (Fri)
by corbet (editor, #1)
[Link] (1 responses)
We have no desire to discriminate against any distribution (or other project for that matter). I do follow the openSUSE lists; I'm even running openSUSE on my desktop machine. I'll look harder for potential topics in the future, I guess...
Posted Sep 2, 2016 13:47 UTC (Fri)
by StefanBr (guest, #110916)
[Link]
[opensuse-factory] Road-map for openSuse 13.3?
which went on till end of may, so for more than a month.
In May 2015, during osC 2015 Richard Brown did two relevant presentations, slides available here:
The ML discussion continued here:
The name Leap appeared really late in the cycle, see here for a summary (2015-06-30):
AFAIK, the *name* was selected by the openSUSE board after the ML discussion.
The availability of Leap was also the result of the SLES sources being available in the OBS, someone picking up what was available and creating a POC, and naming the POC openSUSE:42.
So the (apparent) lack of discussion prior to openSUSE:42 is due to the fact someone just did the first setup, the result which was very welcomed by most openSUSE community members.
A lot of discussion likely happened during osC 2015 (unfortunately I did no attend), and future directions where discussed again during osC 2016. Slides and videos are public.
Posted Aug 29, 2016 21:28 UTC (Mon)
by johannbg (guest, #65743)
[Link] (12 responses)
Now you could make a realistic argument by saying the Debian, and derivative distributions have been better at implementing applications or applications stack using DEB formats since it's implementation is less fragmented in Debian and it's derivatives compared to distribution that use the RPM format like Fedora and derivatives vs OpenSuse and derivatives vs distributions that use wait for it... rpm5 <sigh>.
Failure of single chosen package format in the linux ecosystem has now reached new heights and led to even more fragmentation by introducing flatpak and snaps as well as OrbitalApps and highlighted Appimage an existing solution which ultimately wont solve anything for upstream since they will be again forced to chose which format to use and at the same time include/exclude certain set of end users ( unless of course they end up on using the appimage which is afaikt the only that does not require the end user to install anything before being able to be used by the end user )
Posted Aug 29, 2016 21:48 UTC (Mon)
by Wol (subscriber, #4433)
[Link] (1 responses)
And as I understand it, a lot of rpm hell was down to the fact that SuSE is rpm-based, but is NOT a Red Hat derivative. Indeed, it pre-dates Red Hat!
So a lot of the packaging decisions for SuSE rpms were incompatible with similar Red Hat decisions. Seeing as pretty much every apt/deb-based distro is a Debian derivative, this problem arose to a much lesser extent. .deb files happily installed on pretty much every Debian derivative, while the same could not be said for rpms - a SuSE rpm would blow up on Red Hat, and vice versa.
Cheers,
Posted Aug 29, 2016 22:01 UTC (Mon)
by johannbg (guest, #65743)
[Link]
Oh how everyone seem to love fragmentation which brings in choices and the self inflicting wounds in the process.
Posted Aug 30, 2016 15:54 UTC (Tue)
by cate (subscriber, #1359)
[Link] (9 responses)
Package format and data is a lot more than an ar or cpio. And tracking files, choices etc. is also an important step.
Two different histories and designs.
Posted Aug 30, 2016 20:06 UTC (Tue)
by BenHutchings (subscriber, #37955)
[Link] (3 responses)
When it comes to building packages, rpmbuild provides more functionality than dpkg-dev. Unfortunately, this is largely implemented in poorly documented and often distribution-specific macros. On the deb side, debhelper provides all that functionality and more, with good documentation and without the splintering between distributions.
Posted Sep 1, 2016 11:08 UTC (Thu)
by ovitters (guest, #27950)
[Link]
There's more room for improvement, but IMO better than it used to be.
Posted Sep 1, 2016 14:44 UTC (Thu)
by jsanders (subscriber, #69784)
[Link] (1 responses)
Posted Sep 2, 2016 6:02 UTC (Fri)
by pabs (subscriber, #43278)
[Link]
Posted Sep 5, 2016 11:17 UTC (Mon)
by mathstuf (subscriber, #69389)
[Link] (4 responses)
The word you're looking for is "idempotent".
> Two different histories and designs.
Agreed. One I can't stand of dpkg is the "oh you installed a daemon, you must want it running all the time right now" behavior when installing packages. No, I'd like to investigate it before running it, TYVM. But no, the only way to get that is to do a non-interactive install which has other consequences (no question about the deplist, angry warning messages, etc.).
Posted Sep 5, 2016 11:24 UTC (Mon)
by liw (subscriber, #6379)
[Link] (3 responses)
Debian policy also defines a way to prevent that: add a /usr/sbin/policy-rc.d that exits with 101.
(I'm not arguing for or against this Debian design decision.)
Posted Sep 5, 2016 12:55 UTC (Mon)
by gracinet (guest, #89400)
[Link]
Thank you, never heard of it! Well, it's true that the immediate start policy has never bothered me so much.
To elaborate a bit, I did a web search, and found this detailed explanation,
which in particular mentions /usr/share/doc/sysv-rc/README.policy-rc.d.gz
All the language there is truly sysvinit oriented, but it seems to work the same way with systemd. That latter post is in Ubuntu context, but, unsurprisingly, it works the same way in Debian itself, I found /usr/bin/deb-systemd-invoke on one of my Jessie systems.
Posted Sep 5, 2016 14:18 UTC (Mon)
by mathstuf (subscriber, #69389)
[Link]
Posted Sep 6, 2016 2:15 UTC (Tue)
by raven667 (subscriber, #5198)
[Link]
Posted Aug 29, 2016 21:37 UTC (Mon)
by kreijack (guest, #43513)
[Link]
On the other side, I feel the "rpm" package is more "user friendly". I never liked the number of the dpkg-*/deb* (sub)commands. Instead rpm, has only one command, its switches are more coherent and linear.
Moreover, to create a RPM package you should edit only a .spec file. With a DEB package, the files to edit are.. a lot....
Posted Aug 30, 2016 12:23 UTC (Tue)
by ovitters (guest, #27950)
[Link]
I'm not paid by Red Hat (though IMO dismissing comments because you consider them biased.. everyone is biased).
RPM hell existed 10+ years ago mainly before the existence of a dependency solver. Then secondly the problem is using random rpms with a dependency on a library with a slightly different .so name. Both problems do not really occur anymore.
I've been upgrading Mandrake into Mandriva into Mageia. According to Wikipedia (https://en.wikipedia.org/wiki/Mandriva) that happened in 2004. I've also moved my system from 32bit to 64bit. All using rpm, though obviously changing architectures isn't super easy.
> redhat, well, as you can see with this bug report, it's run by corporate types who do what they are told
Red Hat is a super big company and I welcome you to prove your claim. Being aggressive against people is NOT a good quality to have! You mention that you're better because you fix bugs faster, this while attacking everyone working for Red Hat. This while you don't know these people!
Posted Sep 1, 2016 12:14 UTC (Thu)
by Tet (guest, #5433)
[Link]
Thus demonstrating that you have no idea what RPM hell is or what caused it (hint: it's nothing to do with rpm/yum/dnf vs deb/apt).
Posted Aug 29, 2016 17:00 UTC (Mon)
by SEJeff (guest, #51588)
[Link] (68 responses)
Having dealt with both debian and redhat based systems over a 10+ year career, I've found pain in both, but consistently more pain with debian based systems. Recovering a broken package manager on redhat is rpm --rebuilddb 95% of the time. It is adding exit 0 to the top of a bunch of gnarly scripts in debian land (no equiv of rpm -e --noscripts unfortunately)
Posted Aug 29, 2016 17:07 UTC (Mon)
by SEJeff (guest, #51588)
[Link] (1 responses)
(hundreds more showed up, I just picked 2)
And these are the files in question many packages lay down that are often poorly written and fail hard entirely breaking apt:
https://www.debian.org/doc/manuals/debian-faq/ch-pkg_basi...
Posted Aug 31, 2016 2:26 UTC (Wed)
by guillemj (subscriber, #49706)
[Link]
Yes, the user should be given enough dynamite to blow off their system, but this one is just giving them the dynamite and the plutonium, together. The correct solution to these problems is always to report the issue, and get a fixed package to upgrade to, and *then* remove it. Of course that might not be realistic on some environments or by some providers. :( The error message in these cases should explain that, and that's something I need to fix first. The point is that if there are removal maintainer scripts they are there to perform cleanup or similar, so if you do not run them or skip bogus ones (and their roll-back maintainer script calls) by ignoring their exit codes you might end up with a crufty system at best or broken one at worst anyway, at which point it might actually seem preferable that the admin has to open, read and edit the script itself (obviously many will simply remove it). In the end I guess I'll just add the option and let users destroy their own systems, which is what will be advised all over the Internets, blaming the package manager in the process anyway, oh well. (And on that front I'd like to add some kind of tainting tracking so that if you do any of such things then when the system does not work and the user reports the issue, we can know that, well, they "voided their support warranty" and they get to keep the pieces together, if the maintainer does not feel like dealing with it, https://wiki.debian.org/Teams/Dpkg/Spec/TaintedDatabase.)
And of course there's a single case were dpkg cannot currently safely recover from some types of broken prerm maintainer scripts, and that one is also covered in the the FAQ (https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Can_dpkg_be_tol...).
But ideally, packages should have no need to contain any maintainer scripts, which would remove this problem entirely, and that's a direction we want to go in the future, see https://wiki.debian.org/Teams/Dpkg/Spec/DeclarativePackaging.
Posted Aug 29, 2016 17:17 UTC (Mon)
by tao (subscriber, #17563)
[Link] (8 responses)
I guess you might end up in such situations if you do stuff like forcing installation, installing packages from other distributions manually using dpkg, or something, but other than that things have worked well. Admittedly I've only used Debian since Hamm.
Posted Aug 30, 2016 7:30 UTC (Tue)
by nhippi (subscriber, #34640)
[Link] (7 responses)
I've certainly had postinst/postrm scripts leave packages in terrible state, where the only recourse is to edit an exit 0 to scripts under /var/lib/dpkg/info. Sometime due to my own packaging mistakes, sometimes because using unstable. But in less used packages these errors do slip to released packages too. The major failing case is when upstream changes configuration file format and maintainer tries to write a conversion script from old format to new. Maintainer scripts are a major tripwire without an easy recovery path. And one of the reasons I wouldn't recommend distributing third party software in .deb files. It would be less painful if there was a built-in rollback mechanism...
Posted Aug 30, 2016 13:36 UTC (Tue)
by imMute (guest, #96323)
[Link] (6 responses)
I work at a place where we use debian jessie in a commercial product. I implemented automatic rollbacks by using btrfs snapshots and a wrapper script around apt that ties in to systemd's system-upgrade.target. If the script detects that apt failed for any reason, it rolls back to the snapshot it made at the beginning of the upgrade run. It's an offline upgrade, so two reboots are necessary to go through the process, but it's pretty darn failsafe.
I think part of the reason that getting something like that built in to apt is going to be hard is because everyone wants to do it slightly differently. Does /home get rolled back too? What about logs? Should it depend on what is mounted where? If it does, which mounts get rolled back? Are updates does online or offline? What determines when old snapshots are cleared out? There are plenty of other design decisions that would go into such a feature and there is no One True Answer to them, so a successful product would have to address all of them. :(
It's got some warts but those are mainly due to my inexperience with these kinds of things - I encourage anyone who wants this kind of a system to try to implement it themselves, it's fairly simple to understand in the grand scheme of things.
Posted Aug 30, 2016 13:58 UTC (Tue)
by tao (subscriber, #17563)
[Link] (4 responses)
Logs shouldn't be rolled back (you need to be able to see *why* a rollback was done).
The rest of your questions make sense though.
Posted Aug 30, 2016 16:17 UTC (Tue)
by juliank (guest, #45896)
[Link] (3 responses)
Posted Aug 30, 2016 18:11 UTC (Tue)
by tao (subscriber, #17563)
[Link] (2 responses)
Posted Aug 30, 2016 19:06 UTC (Tue)
by farnz (subscriber, #17727)
[Link]
So if I supply my third party software to you as Debian packages, where should it go? The obvious location is /opt with dependencies on system packages to provide my runtime environment, but you seem to be saying that my choice of package file format affects the acceptable install locations...
Posted Aug 30, 2016 19:07 UTC (Tue)
by juliank (guest, #45896)
[Link]
You can also look at the FHS: The directories /opt/bin, /opt/doc, /opt/include, /opt/info, /opt/lib, and /opt/man are reserved for the local system administrator; packages installing to /opt must install to /opt/<package> or /opt/<provider>.
So, if you want to, you could exclude /opt/{bin,doc,include,info,lib,man} from a rollback as those are reserved for local use, but not the <package> or <provider> subdirectories (practically anything else).
[1] the less package managers on one system, the better
Posted Aug 30, 2016 16:09 UTC (Tue)
by niner (subscriber, #26151)
[Link]
That's the nice thing about being behind other distros. You can copy their solutions without having to make the same mistakes in the beginning.
Posted Aug 29, 2016 17:34 UTC (Mon)
by amacater (subscriber, #790)
[Link] (3 responses)
I've been working with Red Hat for only about 10 years. You break Red Hat - you reinstall it from scratch because nobody can repair it. You try to upgrade Red Hat or CentOS from one major version to another - official advice prior to 6-7 is to reinstall from scratch. 6-7 is a hacky script not guaranteed to work and absolutely guaranteed to fail with third party repositories [Is there anyone without packages from EPEL / rpmforge in its multiple incarnations.]
RPM - nobody gives a care. Nobody knows who has the official version: SUSE has their own variant. [Besides which - Red Hat have produced at least three major buggy versions whihc broke systems with major incompatibilities. Yum - built to get round some of the obvious deficiencies - is now deprecated in favour of DNF]
dpkg - at least Debian will care and Ubuntu will take up the fix once Debian has produced it.
Posted Aug 30, 2016 15:17 UTC (Tue)
by ovitters (guest, #27950)
[Link] (2 responses)
That RHEL support policy, it is not related to RPM. With RHEL, any RHEL version still gets big updates (6.0, 6.1, etc). Those upgrades work just fine and they change loads between those releases. A RHEL version is supported longer than likely your hardware.
I dislike that they don't support upgrades, but for RHEL it isn't that big of a deal. It is important for Fedora and so on. For those it works just fine.
> Red Hat have produced at least three major buggy versions
What do you mean? I don't recall these versions.
> Yum - built to get round some of the obvious deficiencies - is now deprecated in favour of DNF
DNF is pretty much YUM but quicker.
You also didn't explain "nobody gives a care".
Posted Aug 30, 2016 18:55 UTC (Tue)
by amacater (subscriber, #790)
[Link]
In the days when you could update Red Hat - version 3 to version 4 broke RPM, version 8 to version 9 broke RPM then, I think, they managed to break RPM between releases of RHEL.
To the point where at one point, Red Hat was built with one version of RPM but shipped with a buggy one that wouldn't install properly. It's ancient history now - but I've been using Red Hat occasionally since about 1994 and professionally since about 2006.
No one gives a care about RPM? - SuSE / Mageia / Red Hat have differing versions and differing functionality. At one point the original maintainer of RPM tried to create a new version (version 5??) and to claim that his was the only canonical RPM. ...
There is only one dpkg - Ubuntu has essentially taken the Debian version and all the Ubuntu derivatives likewise thereafter. Apt was designed as a nicer front end to sit on top of raw dpkg and that also led to aptitude / synaptic and others. Dpkg hasn't changed much in many years.
[Disclaimer: I did suggest the name apt - which settled a flamewar at the time about deity - but the list for apt development is still called deity-devel, I think].
[Debian predates Red Hat fairly considerably :) In the early days, it is rumoured that Bob Young considered dpkg as a possible contender for package manager]
Posted Aug 30, 2016 20:23 UTC (Tue)
by johannbg (guest, #65743)
[Link]
Upgrades in Fedora never worked reliably and never can work reliably since you can never test all the combination of installable packages in the distribution ( let alone in conjunction with 3rd party components in repository's like rpmfusion which most end users have ).
In fact upgrades were never "officially" supported until Will Woods and Jeremy decided to start playing with writing pre-upgrade or pre-fail as users called it, which today has been rewritten and renamed so many times by Will that everyone has lost count.
Posted Aug 29, 2016 18:52 UTC (Mon)
by josh (subscriber, #17465)
[Link] (11 responses)
No, because dpkg actually handles that exact problem, so that a package can't become unfixable. If you try to upgrade a package, and the maintainer script for the old version of the package fails, dpkg will run the corresponding maintainer script for the new version and expect it to handle the logic for the old version. So as long as you have a fixed package, your system can't get completely broken in a way that dpkg can't upgrade to fix.
Posted Aug 29, 2016 19:04 UTC (Mon)
by SEJeff (guest, #51588)
[Link] (7 responses)
This is a failing of the entire PPA idea, but it was a technical failing of dpkg in and of itsself that I had to dig around and manually futz the postinst / prerm scripts. This is simply rpm -e --noscripts or rpm -ivh --noscripts in Redhat.
Posted Aug 29, 2016 19:19 UTC (Mon)
by josh (subscriber, #17465)
[Link] (2 responses)
Posted Aug 29, 2016 20:09 UTC (Mon)
by rahulsundaram (subscriber, #21946)
[Link] (1 responses)
Bad packages can exist in any format. If it is not that hard to provide some options for workarounds, there is no reason to be idealistic. It isn't any more of a problem than a force option or an option to ignore dependencies etc.
Posted Aug 31, 2016 2:43 UTC (Wed)
by guillemj (subscriber, #49706)
[Link]
Posted Aug 29, 2016 19:42 UTC (Mon)
by dskoll (subscriber, #1630)
[Link] (2 responses)
Well, yes, I should qualify this by saying I only ever run Debian Stable. So I guess I'm insulated from a lot of the QA problems that might be found in the more bleeding-edge releases.
Posted Aug 30, 2016 1:59 UTC (Tue)
by jwarnica (subscriber, #27492)
[Link] (1 responses)
That doesn't say anything about the packing format and toolings ability to handle random packages from the wild, or the general stability of "the wild".
Posted Aug 30, 2016 19:00 UTC (Tue)
by dskoll (subscriber, #1630)
[Link]
Given the ability of both Yes, there are ways around this such as disabling scripts and/or running everything in a sandbox, but they're not practical in production situations.
At this point, you need to rely on the quality and integrity of the distribution provider and not so much on the particular packaging tool used. Nevertheless, whichever packaging tool is used should have bugs found via fuzzing fixed in a reasonable timefarme.
Posted Aug 29, 2016 21:27 UTC (Mon)
by kiko (subscriber, #69905)
[Link]
(Tangentially, it's vexing we have yet to fully flesh out the identity verification piece in Launchpad.)
You're are, it's true, running third-party installation code as root. Yes, if the packages are really poorly implemented, you can easily get into a certain class of dependency hell. And absolutely yes, it's a stopgap until we have a better way of distributing third-party software in a simple and secure way. But I think faced with the tradeoff above, and given how well PPAs have worked in general for distribution of updated or unpackaged software, they are a net win.
Posted Aug 29, 2016 22:25 UTC (Mon)
by micka (subscriber, #38720)
[Link] (2 responses)
Then I must incorrectly what we are talking about. Because I occasionally run into prerm scripts that fails on package upgrades. It makes the upgrade fail and even the remove fail. Those are the time I msut go and edit the script in the dpkg scripts base to make it return early and finish the upgrade.
So what I describe _does_ happen. One solution is: I misidentifed the problem you and the GP are talking about. Is that the case ?
Posted Sep 7, 2016 1:41 UTC (Wed)
by dbnichol (subscriber, #39622)
[Link] (1 responses)
By the way, good luck figuring out the maintainer script semantics just by reading there policy manual. I was told that the above diagram was made by someone writing a program to parse the states in dpkg.
Posted Sep 7, 2016 7:03 UTC (Wed)
by micka (subscriber, #38720)
[Link]
while (pre-rm-1.2.x failed)
Posted Aug 29, 2016 18:56 UTC (Mon)
by dskoll (subscriber, #1630)
[Link]
Have you seriously never had the misfortune of impossible to uninstall (or reinstall, or install) packages that required manually editing stuff under /var/lib/apt/*?
Nope. Never. Running Debian on tens of systems at home and at work for a decade, and that has never happened to me.
Posted Aug 29, 2016 20:25 UTC (Mon)
by Darkmere (subscriber, #53695)
[Link] (38 responses)
Other massive failures like maintainer scripts failing if the user inside a root doesn't exist _outside_ it ( dpkg really doesn't do roots very well).
Posted Aug 29, 2016 20:37 UTC (Mon)
by SEJeff (guest, #51588)
[Link] (21 responses)
preseed is awful (only autoyast is worse). Canonical attempted to repair this by writing kickseed but that ultimately failed.
abrtd + faf are amazing for being able to gather any segfault / thing to drop core on thousands of nodes and have a central reporting place. I think Ubuntu has a trimmed down simplified version of this with the apport retracer bits, but faf is generic enough it would be quite trivial to write a dpkg plugin and just have one to rule them all.
rpm -V so hard! I recall several years back (5 or 6, maybe more?) Debian Packaging Policy didn't require including the bits to run debsums, so there were literally packages in the archive that didn't work with debsums. Unbelievable to me. I know there was a huge bit of work from Canonical to fix all of the packages missing them and then the policy changes to require it. However, you still can't get the information you can get from rpm -V from any of the debian tools that I'm aware of. Simply put, a silly hack like this thing I wrote for fun (http://www.digitalprognosis.com/opensource/scripts/restor...) is absolutely impossible to do on any dpkg based system.
I could literally go on quite a while, but find the Redhat tends to be a better overall distro. Yes Debian Stable is really great for certain use cases, but for what I do (HPC / Finance), Redhat based distros simply blow it out of the water.
Posted Aug 29, 2016 21:36 UTC (Mon)
by Darkmere (subscriber, #53695)
[Link] (12 responses)
Having an installed system and being unable to verify the installed packages post-installation, because they only sign the list of hashes of all packages. So even if you keep a backup copy of all packages you install, you cannot verify the content or state of them.
It requires a special kind of madness to design this system.
Posted Aug 29, 2016 22:03 UTC (Mon)
by amacater (subscriber, #790)
[Link] (11 responses)
You know too, that the installation process checks and verifies, just as on Red Hat?
Essentially, pretty much what you can do on Red Hat you can as easily do on Debian.
Posted Aug 30, 2016 10:01 UTC (Tue)
by niner (subscriber, #26151)
[Link] (10 responses)
So dpkg seems nowadays been able to do a subset of the checks rpm does though arguably it's the most important part. It even uses rpm's output format.
Good to see dpkg catching up there.
Posted Aug 30, 2016 10:06 UTC (Tue)
by Darkmere (subscriber, #53695)
[Link] (9 responses)
apt (not dpkg) only signs the list of hashes of all packages.
There is no signature on the metadata of each package, which means that the list of sizes, permissions, and checksums are not signed.
So you can't go from "I have foobar-1.0_4.deb installed", to "Is it possible to verify that the files I have are the ones that Debian shipped?"
in an RPM package, the metadata is signed for each package. In deb, dpkg doesn't even support signatures, that's all done on the APT-layer. Officially, this is to be able to revoke signature on a file.
Unofficially it seems to be designed to make it easier to backdoor Debian & Ubuntu systems.
Posted Aug 30, 2016 20:24 UTC (Tue)
by BenHutchings (subscriber, #37955)
[Link] (7 responses)
There is a small advantage that signed packages get you here, which is you could maybe pull the original package out of a cache on the suspect system and verify its signature before using it as a reference. Without package signatures, you would have to download it again from a trusted archive. Is that what you're getting at?
Posted Aug 30, 2016 20:45 UTC (Tue)
by Darkmere (subscriber, #53695)
[Link] (3 responses)
package contains a list of filenames + hashes + metadata
File contents can be validated by the hash, integrity of the list of the hashes can be verified by itself.
The problem is that on a given debian system, you cannot go from "this package is 1.1_deb33" installed, and come to the answer of the question "Have it's files been maliciously tampered with".
There is no historical list of these signatures, once a new package is uploaded to the debian mirrors, you cannot verify any older package.
There is also no policy of not fucking around with the contents of binaries on disk -after they have been installed- by either this package, or some other package. It happens quite often that post/pre-install scripts start to mess with the files on disk.
The reasoning that Debian choose to blame was the fact that "an older package that contains serious bugs should not be trusted if a new has been published". A false claim if any ( you could simply not trust the on-file signature for installation purpose. )
What frustrates me is that once a system is no longer 100% up to date, you cannot validate that the files you have installed are unmodified.
Posted Aug 30, 2016 21:22 UTC (Tue)
by BenHutchings (subscriber, #37955)
[Link]
That sounds quite good, though its usefulness depends on exactly what's being hashed. (For example, does the signed hash include the package name and version?) That's not longer true for at least Debian, since historical packages and the corresponding signed package indexes are accessible through snapshot.debian.org. However it's not a totally straightforward procedure to download and verify a binary package from there (and the debsnap tool doesn't), let alone to check all the packages supposed to be installed on a suspect system.
Posted Sep 2, 2016 14:29 UTC (Fri)
by yoe (guest, #25743)
[Link] (1 responses)
Posted Sep 4, 2016 11:48 UTC (Sun)
by Darkmere (subscriber, #53695)
[Link]
And as someone elsewhere stated, apparently dpkg does support signed binaries, just that Debian doesn't use it.
Debian, when given two choices would make two optional and a third mandatory.
Posted Aug 30, 2016 21:19 UTC (Tue)
by lsl (subscriber, #86508)
[Link] (2 responses)
There are other kinds of unwanted modification, like accidental corruption. Think of running "make install" where the Makefile unexpectedly defaults to a prefix of /usr. Or the unbounded joy of things like pip/rubygems/… happily blowing away system packages.
That's where rpm -V style verification can be very useful.
Posted Aug 30, 2016 22:03 UTC (Tue)
by BenHutchings (subscriber, #37955)
[Link] (1 responses)
Posted Aug 30, 2016 22:20 UTC (Tue)
by guillemj (subscriber, #49706)
[Link]
Posted Aug 30, 2016 22:26 UTC (Tue)
by guillemj (subscriber, #49706)
[Link]
Posted Aug 30, 2016 4:41 UTC (Tue)
by voltagex (guest, #86296)
[Link] (7 responses)
Posted Aug 30, 2016 13:56 UTC (Tue)
by imMute (guest, #96323)
[Link]
The biggest problem I had with the whole process was not actually with preseed itself, it was with repackaging the installation media. I was simply trying to take the netinstall image, slip in my own preseed file, and repackage it for use with a USB stick. I found a couple sets of instructions on how to do that and the repackage step either would use a command that doesn't work on Jessie, or would produce an image that wouldn't boot. I eventually got it mostly working using an Ubuntu 14.04 system to run the repackage command.
It was about a year ago when I attempted this, and we gave up since cloning the entire drive was a faster way anyway - things may be different these days.
Posted Aug 30, 2016 15:41 UTC (Tue)
by SEJeff (guest, #51588)
[Link] (2 responses)
On the other hand, preseed is more murky in comparison and the documentation (used to be) horrible:
# This is how to make the installer shutdown when finished, but not
The redhat equivalent (https://access.redhat.com/documentation/en-US/Red_Hat_Ent...) is halt, just like the shell command. This is a relatively obscure reference, but in general as a sysadmin to write a preseed from scratch, you have to understand a lot of how debian-installer works.
As a sysadmin to write a kickstart from scratch, you need a lightly templated shell script with a few special stanzas. It is just so much easier.
Posted Aug 31, 2016 12:05 UTC (Wed)
by pizza (subscriber, #46)
[Link] (1 responses)
You don't even need to write it from scratch -- Do a single installation with the rough (or exact) settings you want, and as part of the installation it'll generate a kickstart file that corresponds to your installation choices. Customize it to your heart's content with addtional packages and your postinstallation scripts, and go to town.
(Kickstart is a wonderful feature. I've been using it since the RHL7 days; every single system we had in production could be completely recreated automatically; stick in a floppy and come back in a couple of hours...)
Posted Aug 31, 2016 13:53 UTC (Wed)
by SEJeff (guest, #51588)
[Link]
Posted Aug 30, 2016 18:32 UTC (Tue)
by edgewood (subscriber, #1123)
[Link]
Posted Aug 30, 2016 19:05 UTC (Tue)
by dskoll (subscriber, #1630)
[Link]
Oh, hey, I love Debian. But let me say this: I hate, hate, hate preseed with a bitter, burning passion. And working with d-i is also an exercise in pain... lots of twisty shell scripts, Perl scripts, C programs, and magical run-parts invocations without any damn clue how it all fits together.
Pressed: no documentation. Extremely fiddly. Incredibly long edit-test cycle (you basically have to make new boot media or PXE images, boot the thing, see what breaks, rinse, repeat.)
And the worst part (though I think this may have been fixed... not sure) was that
some of the answers were locale-sensitive. So if you had a user who picked an
unexpected locale, all the preseed answers would be borked.
Posted Aug 30, 2016 21:30 UTC (Tue)
by seyman (subscriber, #1172)
[Link]
One of the things I've alway appreciated when using kickstart is that a kickstart file is always generated during an installation of Fedora/RHEL/Centos (found in /root/anaconda-ks.cfg). This allows you to perform an install, grab the kickstart file and be 99% done.
Posted Aug 30, 2016 4:29 UTC (Tue)
by voltagex (guest, #86296)
[Link] (12 responses)
These should fail on the package builder servers / reproducible build servers, yes?
If you've got any examples of these in stable or testing, I would have thought you could raise a bug - let me know if you need a hand.
Posted Aug 30, 2016 14:17 UTC (Tue)
by Darkmere (subscriber, #53695)
[Link] (11 responses)
Prime one is just to grep for 'mawk' rather than 'awk' in all the packages. Perl is another favourite, that will be present in all "normal" debian systems, since apt requires it, but if you build a new root (say a container) then, suddenly...
Posted Aug 30, 2016 16:25 UTC (Tue)
by juliank (guest, #45896)
[Link] (10 responses)
Policy 3.5:
Posted Aug 30, 2016 16:49 UTC (Tue)
by Darkmere (subscriber, #53695)
[Link] (4 responses)
This is where debian packaging _shines_ at being obtuse.
base-files is essential
This then causes a hilarity of issues when bootstrapping, because you need to run the preinst configure job for mawk before any others, otherwise "awk" will not point to "mawk", something that dpkg is, by and of itself, uncapable of figuring out.
Posted Aug 30, 2016 18:19 UTC (Tue)
by tao (subscriber, #17563)
[Link] (3 responses)
As far as bootstrapping goes -- debootstrap usually does its job without a hitch -- are you saying that it fails for you?
Posted Aug 30, 2016 18:23 UTC (Tue)
by Darkmere (subscriber, #53695)
[Link] (2 responses)
I _have_ read the chapters, and I've worked around these issues, but they _are_ causing a major painpoint and showing some of the haphazard decisions in the debian packaging system.
The simple fact that a lot of the pre/post-install scripts are larger than several binaries on the system should be a cause of concern for most people.
Posted Aug 30, 2016 22:13 UTC (Tue)
by guillemj (subscriber, #49706)
[Link] (1 responses)
I've covered the situation with the awk virtual package in the following debian-policy bug message <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=759491#156> (see the whole thing for more details), so this is a case of lore that has unfortunately never been transcribed into proper documentation.
Also bootstrapping Debian is outside the realms of debian-policy, it's not covered there, debian-policy assumes that the pseudo-essential set has been fully configured at least once, I've gone into more detail in the following mail <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=767999#157>.
I've also drafted in the recent past a proposal to clean this up <https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap>, which might fix your issues in case you are trying to bootstrap something that is not supported by debootstrap, this "just" needs coordination, consensus and work.
Posted Aug 31, 2016 21:45 UTC (Wed)
by Darkmere (subscriber, #53695)
[Link]
Posted Aug 31, 2016 12:35 UTC (Wed)
by nye (subscriber, #51576)
[Link] (4 responses)
This policy has always upset me. I've never seen any explanation of why it's this way, which is strongly needed because on the face of it it's clearly a splendidly terrible idea. Normally when an idea is obviously bad, I expect to see some reasoning about why it's actually less bad than the alternatives.
I presume those reasons do exist, because nearly nothing happens in Debian without a reason, but whenever I've seen people ask about this (I'm too afraid to do so myself) the conversation seems to go:
Q: So why is this policy?
Posted Aug 31, 2016 23:16 UTC (Wed)
by anselm (subscriber, #2796)
[Link]
Presumably the idea is to avoid burdening lots of packages with long and largely identical lists of dependencies to “essential” packages. These tend to bog down dependency solvers, which can be a hassle on machines that are slow(ish) and/or don't have vast amounts of RAM, like the sort of PC that was current when Debian was new in the mid-1990s.
This policy has been around for a very long time, and in 2016 we could perhaps afford the explicit dependency lists more easily than we used to in 1995, but it is safe to say that nobody is looking forward to fixing this in tens of thousands of packages.
Posted Sep 1, 2016 12:48 UTC (Thu)
by jwilk (subscriber, #63328)
[Link]
Posted Sep 3, 2016 3:02 UTC (Sat)
by guillemj (subscriber, #49706)
[Link] (1 responses)
* They are needed by dpkg itself when performing operations on packages, on itself included (although this surface is being shrinked).
Thus:
* They have to be very hard to remove, requiring force options so that they are "guaranteed" to be always present.
Hope this clarifies things a bit. :)
Posted Sep 5, 2016 12:23 UTC (Mon)
by nye (subscriber, #51576)
[Link]
Thanks, this is the part that seems to be missing: that there is no way to declare a dependency that can be satisfied by a package being unpacked, but not configured. I wonder if there's any possibility of having this explicitly spelled out in section 3.5? (I have no idea how hard it is to get an explanatory note added to the policy document; maybe it's straightforward or maybe it's the same process as changing the policy itself.)
The issue of dependency loops is the only other explanation I've seen mentioned, but in itself it doesn't seem sufficient given that it could be solved with some special case rules much simpler than the current special case prohibition on declaring them as dependencies, or possibly is already covered by the other properties of essential packages.
It's unfortunate that this special dependency status wasn't created in the first place, rather than the rule in 3.5, because at least having that information declared makes changing the essential set (or creating ultra-small derivatives like the old Emdebian Crush) a more tractable problem, rather than pretty much requiring an audit of every package. Obviously I understand that hindsight is 20:20 and that changing it now would be a mammoth job without immediate benefit.
Posted Aug 30, 2016 22:33 UTC (Tue)
by guillemj (subscriber, #49706)
[Link] (2 responses)
That should (in principle!) not happen when using either --root or --instdir, but if you have a reproducer I'm very interested in a bug report, or a short recipe, so that I can get it fixed. Thanks.
Posted Aug 31, 2016 21:58 UTC (Wed)
by Darkmere (subscriber, #53695)
[Link] (1 responses)
On a system (well, container) created with `debootstrap --include=debootstrap stable <something>`, run:
This means that the building container doesn't have `dbus` installed and the chroot inside does. Wich causes debootstrap to fail.
Posted Aug 31, 2016 22:44 UTC (Wed)
by guillemj (subscriber, #49706)
[Link]
Posted Aug 30, 2016 3:38 UTC (Tue)
by rsidd (subscriber, #2582)
[Link]
No, I have never edited anything in /var/lib/apt manually. I have had broken packages, thanks specifically to unmaintained PPA or other third-party packages that got orphaned after a dist-upgrade leading to version conflicts. All I had to do was disable the PPAs (editing files in /etc/apt, not /var/lib/apt), remove all offending packages, and reinstall what I needed.
I get that Red Hat is very different now from back when I used it. I am just surprised at the quoted response from the rpm bugs team.
Posted Aug 29, 2016 20:47 UTC (Mon)
by roc (subscriber, #30627)
[Link]
Posted Aug 30, 2016 18:31 UTC (Tue)
by juliank (guest, #45896)
[Link]
The github repository has a rpm-4.13.x branch, the latest (pre-)release of that being 4.13.0-rc1. This one is also listed on the RPM website.
The Fedora package is based on that pre-release. The f22 thing you linked to cherry-picks a few commits from the rpm-4.13.x branch (or 4.13.0 pre-release), hence the patch file names.
Given that this is all fairly obvious, I'm not sure how in the hell one can get so confused.
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
I expect to hear about ruby gem hell so I hear that more often than about pypi ones.
Böck: Multiple vulnerabilities in RPM – and a rant
> own sandbox with its own precisely-defined set of packages, reports of package conflicts are probably less likely than
> from the Ruby crowd unless they've also gone down the same road.
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
obredhat, lol
obredhat, lol
obredhat, lol
obredhat, lol
obredhat, lol
Could you please point to examples of this pro-Red-Hat bias you apparently see here at LWN? I don't see it, but, then, we're often blind to our own biases...?
Bias
Bias
Bias
Bias
There is no doubt that the amount of coverage we have is proportional to how open and visible a community's processes are. Fedora and Debian are the most open, so they tend to get the most attention. It's a lot harder to know what's going on inside the others.
Bias
Bias
Consider, for example, the whole openSUSE Leap transition. The Fedora community would have argued about the concept for some time; openSUSE, instead, saw it announced in nearly its final form. So we didn't have a series of articles as the idea took shape; we had only one after it was announced.
Bias
Bias
[2] https://bbs.archlinux.org/viewtopic.php?pid=1149530#p1149530
Bias
Bias
Bias
Bias
Bias
Bias
Bias
It debates *a* *LOT* there.
I can confirm that because the lates public discussions and decisions made me turn away from openSUSE -- after having used the stuff since S.u.S.E 4.2 in 1996 -- and believe me, it needs some time to turn your back on a distribution you used for 19+ years.
So yes, I percieve a clear Fedora/RH bias in LWN.net reporting. [*] It doesn't bother me, though. I don't know if the resources are available to remedy it, and I don't know if the resources would be well spent.
Again, what was public about openSUSE Leap, for example? Once it became public, we covered it, but the decision had already been made at that point.
Bias
Bias
https://lists.opensuse.org/opensuse-factory/2015-04/msg00...
https://speakerdeck.com/sysrich
https://speakerdeck.com/sysrich/opensuse-vision
https://speakerdeck.com/sysrich/the-future-is-unwritten
[opensuse-factory] openSUSE:42
https://lists.opensuse.org/opensuse-factory/2015-06/msg00...
Re: [opensuse-factory] How to name the baby -- A Summary
https://lists.opensuse.org/opensuse-factory/2015-06/msg00...
obredhat, lol
obredhat, lol
Wol
obredhat, lol
Forks are good aha...
obredhat, lol
obredhat, lol
obredhat, lol
obredhat, lol
obredhat, lol
obredhat, lol
Debian and starting daemons automatically on install
Debian and starting daemons automatically on install
Debian policy also defines a way to prevent that: add a /usr/sbin/policy-rc.d that exits with 101.
Debian and starting daemons automatically on install
Debian and starting daemons automatically on install
obredhat, lol
obredhat, lol
The term rpm hell came about due to user experience, not because of cognitive bias. If redhat wasn't corporate linux wedded to rpm they would have moved to apt/deb ages ago
obredhat, lol
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
https://en.opensuse.org/openSUSE:Snapper_Tutorial
Böck: Multiple vulnerabilities in RPM – and a rant
[And apt/aptitude etc. use the same underlying mechanisms and databases]
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
.deb and .rpm packages to invoke arbitrary shell scripts, you've basically lost. A malicious packager can do anything he or she wants, up to and including nuking your entire system.
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
So the correct answer is
get maintainer to create 1.2.(x+1)
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Directly calling various awk implementations, the OpenSSH package calling out to perl in order to re-write the configuration file (And also leading to the OpenSSH server configuration file not being listed as a configuration file. Because it's a special snowflake.)
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
"Currently the only functional check performed is an md5sum verification of the file contents against the stored value in the files database. It will only get checked if the database contains the file md5sum."
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
package contains a signature blob of above list
package manager then saves this in the database
Böck: Multiple vulnerabilities in RPM – and a rant
File contents can be validated by the hash, integrity of the list of the hashes can be verified by itself.
There is no historical list of these signatures, once a new package is uploaded to the debian mirrors, you cannot verify any older package.
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Not a part of the package management tools.
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
dpkg has supported signed .deb for a long time (since dpkg 1.9.0) via the debsig-verify package (check also the --no-debsig dpkg option in /etc/dpkg/dpkg.cfg). You can sign them with the debsigs package.
Debian just does not use signed .deb packages, it uses signed repository indices.
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
# reboot into the installed system.
d-i debian-installer/exit/halt boolean true
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Also not the person you were responding to, but I've been learning preseed as a way to have a reproducible install of servers, both bare metal and VMs. From my perspective, the things I don't like about preseed are:
Böck: Multiple vulnerabilities in RPM – and a rant
So basically doc and improvements to partitioning, I guess.
preseed (was Böck: Multiple vulnerabilities in RPM – and a rant)
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Packages are not required to declare any dependencies they have on other packages which are marked Essential (see below), and should not do so unless they depend on a particular version of that package.
Böck: Multiple vulnerabilities in RPM – and a rant
libc6 is essential
base-files has Pre-Depends on awk
(m)awk is of course _not_ Essential.
libc6 of course _uses_ awk in the setup scripts.
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
>Packages are not required to declare any dependencies they have on other packages which are marked Essential (see below), and should not do so unless they depend on a particular version of that package.
A: Because it's part of our packaging requirements.
Q: So why is it part of our packaging requirements?
A: Because it's policy.
(Perhaps with a side order of "it's always been that way".)
Böck: Multiple vulnerabilities in RPM – and a rant
Um, Policy does explain it:
Böck: Multiple vulnerabilities in RPM – and a rant
Essential is needed in part to avoid unresolvable dependency loops on upgrade. If packages add unnecessary dependencies on packages in this set, the chances that there will be an unresolvable dependency loop caused by forcing these Essential packages to be configured first before they need to be is greatly increased. It also increases the chances that frontends will be unable to calculate an upgrade path, even if one exists.
Böck: Multiple vulnerabilities in RPM – and a rant
* They are needed to boot the system, at least to a point where you can run into some kind of recovery mode (although there's been talk about getting rid of this for the benefit of chroots and similar).
* They are needed by several maintainer script types, which have no other way to declare dependency satisfiability at that point; postrm and config maintainer scripts cannot rely on any of their dependencies being satisfied, and only rely on the implicit set of Essential:yes packages. preinst can only rely on Essential:yes and Pre-Depends being satisfied (but not necessarily fully configured).
* They have to be always functional, regardless of their installation state (even when not fully configured, just unpacked or other transitioning states), and regardless of system crashes/reboots/power-outage/etc midair on dpkg operations (barring filesystem/disk corruption). There's no dependency field to distinguish that relationship, because both Pre-Depends and Depends are too strong. So we'd need to add a new field just to declare that kind of relationship, which means that removing stuff from Essential would probably be similarly burdensome just in a different way, as we'd have to update thousands of packages to switch the field name. (I've not even thought how a partial upgrade from a release that has a package marked as Essential:yes to one that does not, would even work with such fictional field.)
* They are used to avoid dependency loops. This does not have much to do with hardware performance, but more to do with how dpkg handles dependency loops in presence of maintainer scripts, it just picks where to break the loop randomly, so you might get in trouble. Which means dependency loops are always an undesirable thing to get into. The above mentioned properties of Essential:yes removes this problem.
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
`debootstrap stable newroot`
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant
Böck: Multiple vulnerabilities in RPM – and a rant